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ABSTRACT 



This study analyzed differences in perception of "critical 
incidents" in classroom teaching among inexperienced, experienced, and 
award-winning university professors. In past research, expert and novice 
differences in teaching have been attributed to differences in teacher 
knowledge, but the teachers were almost entirely at the elementary or 
secondary levels rather than the post-secondary where instructors have 
subject matter expertise rather than formal training in teaching. Analysis 
was based on a total of 102 questionnaire responses, all from full-time 
university instructors, fairly equally distributed among the three experience 
levels. Each summarized response was coded to characterize the professor's 
concerns and thinking associated with a memorable teaching incident, using 
the categories of: (1) knowledge, which could be information about students 

or about an instructional strategy; (2) processes, as pacing, or awareness of 
student comprehension levels; (3) goals, or desired outcomes; and (4) 
actions, or activities. The study found that, while categories used in the 
coding process could be hypothetically separated, it was frequently difficult 
in practice. There was much overlap between groups in the categories 
suggesting that the same category may have a different context in the 
different stages of a teaching career. Results have implications for faculty 
development activities. The Critical Incident Questionnaire is attached. 
(Contains 26 references.) (BF) 
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Objective 

The objective of this study was to characterize the critical incidents of 
teaching held by relatively inexperienced university teachers (newly hired faculty), 
experienced faculty (more than ten years on the job), and award winning 
professors (university and national awards). 

, Theoretical Framework 

In the past decade educational researchers have taken an ever increasing 
interest in the process of teaching and teacher thinking (e.g., Clark & Peterson, 
1986; Leinhardt & Greeno, 1991). This research has lead to a number of generic 
principles of "good teaching" which include recommendations on how best to plan 
instruction, prepare handouts, stimulate student interest, evaluate learning, and so 
on (e.g., Clark, 1993; Gillett-Karam, 1992). In addition, teaching has been 
recognized as a complex and ill-defined task in which teachers perceive input from 
the immediate environment and utilize it to generate appropriate instructional 
actions. 

Past research indicates differences in the thinking of teachers at various 
stages of their careers (e.g., Calderhead & Robson, 1991; Lienhardt & Smith, 
1985). Experts perceive classroom events differently and use more information 
about learners than do novices (e.g., Calderhead, 1981; Carter, et. al., 1987; 
Carter, et. al., 1988; Copeland, et. al., 1994, Veenman, 1984). Expert and novice 
differences in teaching have been attributed to differences in teacher knowledge 
(e.g., Calderhead, 1988; Peterson & Comeaux, 1987; Shulman, 1986). Expert 
teachers have been found to have elaborate and rich cognitive schemas that 
provide a frame-work, inform, and facilitate action in complex classroom 
situations; overall experts know what to expect in classroom situations. They are 
able to monitor classroom situations (e.g., students), recognize problems, and 
make rapid decisions that meet their teaching goals (e.g., Carter et al, 1987; 
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Leinhardt & Smith, 1985). Indeed, many researchers have suggested that novices 
have less well elaborated schemas than do novices and lack metagcogntive and 
monitoring skills that typify expertise (e.g.. Gage & Berliner; 1984; Gagn6, 1985) 

Unfortunately, the bulk of the research on teacher thinking has been 
conducted in elementary and high school settings; there is still a paucity of 
research addressing post-secondary teaching. There is little doubt that post- 
secondary instruction differs from lower levels in that instructors or professors are 
typically subject-matter-experts and do not have formal training in teaching 
(Ramsden, 1992); thus their knowledge base is markedly different than other 
teachers. 

This paper focuses on differences in perception of "critical incidents" in 
classroom teaching among inexperienced, experienced, and award winning 
university professors'. It seeks to characterize professors concerns and their 
thinking associated with the memorable teaching incidents. 

Method 

This study represents a portion of the data collected for another study 
(Rahilly, in progress). Analysis was based on a total of 102 respondents who are 
full-time university instructors representing three groups. Group one (n = 33) had 
taught for less than ten years, group two (n = 29) had been teaching for ten years 
or more, and group three (n=40) were faculty who had won teaching awards 
(national, provincial, or university). Participants were selected at random from 
published lists of award winning professors, membership directories of national 
organizations such as the Society for Teaching and Learning in Higher Education 
(STLHE), through requests posted on various national listservs concerned with the 
improvement of university teaching, as well as through the assistance of teaching 
centres at several universities across the continent. 
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Participants were contacted by email or regular mail and received a copy of 
a (lengthy) questionnaire. The questionnaire is based on a pilot study (Rahilly & 
Saroyan 1995) and on the critical incident technique outlined by Flanagan (1954) 
and Woolsey (1986). The use of the critical incident technique in conjunction with 
a questionnaire is a multi-method approach, which strengthens the account of 
teachers knowledge and beliefs Kagan (1990). Like all methods, this method is 
imperfect; it is retrospective in nature and does not capture thinking in action. 
However, the critical incident technique is rooted in a phenomenological approach 
and has been shown to yield data which identify the underlying assumptions 
related to thoughts, actions, and beliefs (Brookfield, 1990). In addition, this 
method is strongly based in an individuals' experience it emphasizes events that 
are meaningful to them and allows for the collection of both qualitative and 
quantitative data about classroom teaching and teaching thinking. 

In this study faculty were asked to complete the Critical Incident 
Questionnaire (CIQ) which is basically a description of two classroom incidents; 
one in which they thought they had done an exemplary job of teaching, and one in 
which they felt they had done a poor job. The Critical Incident Questionnaire 
(CIQ) (see Appendix A) was developed based on Flanagan's (1954) original 
description of the critical incident technique in conjunction with Benner's (1984) 
description which emphasizes an incident as one in which the respondent felt their 
intervention made a difference (for the better or worse). The CIQ is composed of 
12 questions (8 open ended) that ask respondents to describe the circumstances 
of the incident (e.g., when, where, kind of class, topic, etc.). A number of 
questions are based on the literature on problem solving (e.g., Hayes, 1989); thus 
respondents are asked to identify their teaching and learning goals, and to identify 
their actions, and to state what they "took" from the situation (i.e., did you learn 
anything). Lastly, the CIQ has one question which asks respondents what they 
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were feeling during the incident. This question was added based on results from 
the pilot study which indicated that emotion was a large element in determining if 
an incident was viewed as being positive or negative. The CIQ for the exemplary 
incident and poor incident are identical with the exception of one question. In the 
exemplary incident respondents are asked "what is it that you know that helped 
you in this incident", while in describing the poor incident they are asked "what is 
it you wish you had known that would have helped you in this incident?" 

Respondents also completed another scale which takes a more quantitative 
approach to the same incidents. This data is not outlined in this study. 

Responses to the CIQ were summarized based on the questions "what 
happened", "why do you think the incident is memorable", and "what do you wish 
you had known at this time". The other questions were also considered and 
responses were included if they contradicted the information outlined in the above 
mentioned questions. Each summarized response was then coded in terms of the 
underlying theme of the incident (which tended to be represented in the question 
"why do you think the incident is memorable" and then coded using the categories 
of teaching outlined in Rahilly and Saroyan 1995. These broad categories are: a) 
knowledge, which refers to what professors say they need "to know" and ranged 
from information about the learners to procedures professors might use in the 
classroom (e.g., knowledge of an instructional strategy such as cooperative 
learning); b) processes which refers to things that the professors said they need 
"to think" about in relation to their teaching (e.g., many professors said they need 
to monitor their pacing as well as to monitor their students' comprehension of 
material); c) goals were the desired outcomes of teaching that the professors felt 
were valuable and ranged from teaching critical thinking to preparing students for 
their next class; and d) actions were the activities that the professors referred to 
explicitly and these included class discussion, questioning, labs, and so on. 
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Results 

Without a larger sample size it is difficult to find similar situations in the 
summaries of the critical incidents or memorable events of university faculty. If 
we temporarily (coded results appear in the next section) abandon all traditional 
notions of quantitative analysis without entering the rigorous world of qualitative 
analysis, there are some interesting observations. Basically, the results presented 
in this section involve reading over the summaries in the same way that one might 
look through a portfolio or a photo album and walking away and telling someone 
what you just saw. We recognize that in reporting this, we are distancing 
ourselves from each "photo" or "work of art" (i.e., teaching incident) and the 
analysis is based on the theory and knowledge base of the reader (i.e., the 
authors). To offer an analogy which emphasizes the strengths and limitations of 
this approach, one could look at the portfolio of a famous photographer like Ansel 
Adams and report that all the photo's are in black and white, or are mostly of 
trees and mountains, or one might report on the perspective or lighting. The 
possibilities are unlimited, and there is an argument that can be made to support 
each view of the same works. With aN of these limitations in mind, here are some 
observations we offer for discussion and for feedback (see Table 1 and Table 2). 

Recognizing the limitations of the method outlined above, each of the 
summarized teaching incidents was coded based on the themes of teaching 
(knowledge, processes, goals, actions). Table 3 represents the percentage (due to 
unequal n's in the groups) of the summaries falling into each category. As this 
paper represents only a portion of the data in the project, quantitative comparisons 
have, as of yet, not been carried out. 
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Poor Teaching 


GrouD 


Observation 


Inexperienced Professors 


Situations in which their credibility and knowledge 
base are challenged or in which they reveal their 
limitations 


Experienced Professors 


Situations in which they have a plan or have taken a 
risk (e.g., to do something different) and it doesn't 
work. 


Award Winning Professors 


Situations when they become frustrated (e.g., 
enthusiasm not shared or unable to be enthusiastic) 
and are surprised at revelations about the learners 
(e.g., background preparation). 



Table 1 : situations reflecting poor teaching 





Exemplary Teaching 


Inexperienced Professors 


Situations when they plan, carry out their plan of 
teaching, and are pleased with the results 


Experienced Professors 


Situations that involve specific teaching goals (using 
a variety of activities) and the goal is met (e.g., get 
them interested, get them to synthesize, etc.) 


Award Winning Professors 


Situations in which they could be flexible and 
change their teaching based on their instincts and 
teach in a way they "know" will work from past 
experience (e.g., particular examples, methods, etc.) 



Table 2 : situations reflecting exemplary teaching 
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Discussion & Conclusion 

Teaching is recognized as a complex process involving a multitude of 
factors, some of which have been addressed in this paper. The categories used in 
the coding process can be hypothetically separated, however, often when applying 
them, it is difficult to tease them apart. For example, in an account of teaching in 
which the professor described getting muddled in the content because s/he didn't 
plan, there are two issues (content knowledge and planning). Similarly, in 
accounts of teaching involving emotion, it is difficult to attribute the entire incident 
to the emotion of the professor, unless they describe that it was there emotion 
(e.g., enthusiasm) that dominated the incident, and not other aspects of teaching 
at that time. In addition, it is important to recognize that the way in which each of 
these categories may be attributed to the teaching of inexperienced, experienced, 
and award winning professors may be slightly different. For example, struggles 
with content differ across the career of a professor (e.g., dealing with 
fundamentals in a field vs. new ideas and research). Lastly, while it is tempting to 
attribute differences in teaching based on these categories, these data represent 
self-report of classroom incidents and include the professors opinion as to what 
they did that made a difference (for better or for worse). So, for example while 
award winners do not mention knowledge of pedagogy in their exemplary 
incidents of teaching, the exemplary incident may well involve a good deal of 
pedagogy that the professor takes for granted. 

Looking over Table 3 it is clear that while there are trends in the reported 
critical incidents of poor and exemplary teaching, these categories of teaching 
occur in all groups. For example, even the most experienced or highly recognized 
professor may struggle with content. It is also interesting to note that while it was 
more frequent among award winners, flexibility and pedagogical content 
knowledge occurred in each of the groups. This trend confirms much of the 
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Category Inexperienced Experienced Award Winning 

Professors Professors Professors 





Poor 


Exemo 


Poor 


Exemo 


Poor 


Exemo 


Knowledge-Content 


19 


- 


17 


4 


11 


- 


Knowledge- Learners 


19 


4 


13 


- 


26 


6 


Knowledge- Pedagogy 


14 


35 


30 


19 


17 


17 


Knowledge- Ped. Cont. 
Knowl 


- 


13 


5 


19 


3 


26 


Knowledge-Role 


- 


- 


5 


- 


- 


3 


Knowledge-Strategy 


- 


- 


5 


- 


- 


- 


Process-Emotions 


19 


7 


13 


4 


26 


6 


Process- Monitoring 


29 


30 


5 


18 


11 


14 


Process-Flexibility 


- 


4 


8 


4 


- 


17 


Goals 


- 


4 


1 


32 


6 


9 


Actions 


- 


- 


- 


- 


- 


3 



Table 3 : Percentage of the summarized incidents of teaching by teaching category 
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research in cognitive psychology that has indicated that as people move from the 
status of novice to expert, changes in thinking occur (e.g., Ericsson & Smith, 
1991; Lesgold, 1988) and supports the work of researchers like Leinhardt and 
Greeno (1991) that addresses planning, and classroom routines. 

The data also have many implications for faculty development activities. 
Looking at the exemplary teaching incidents it is clear that these different groups 
of faculty would benefit from collective efforts to improve their teaching; each 
group has something too offer the other! The challenge for faculty developers is 
how to tap these resources. Looking at the data representing the poor incidents 
for each of the group may be an interesting place to start looking at possible 
faculty development activities for each of these groups. For example, 
inexperienced faculty could be benefit from learning how to make meaning of the 
cues they glean from students, experienced faculty may benefit from interventions 
dealing with pedagogy (e.g., how to plan, and still try to be flexible), and award 
winning faculty might benefit from interventions to give them information about 
their learners and to deal with the emotions of teaching (or teaching as an award 
winner). 
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EXEMPLARY Incident 



The purpose of this section of the questionnaire is to get a sense of what occurred 
during this incident when you felt you did an exemplary job. If you can't think of 
an incident easily, please take a moment to think of the last time you felt 
challenged as a teacher. Or you may think of a time in recent years when, due in 
part to your actions or efforts, things went very well. 

Once you have selected an incident, fill out the questions below. If you can't 
recall specifics, then please do your best to answer the question. If the question 
does not apply, then please indicate this in the space provided. 

1. Approximately how long ago did this incident occur? 

This semester Last semester Last Year 

Other (specify): 

2. What was the situation? 

a. How many students were in the class? 



b. Title of the course (approximate): 



c. Required course: , Elective course: , Mixed: 

d. How many times had you previously taught the course? _ 

e. Type of students? Undergraduate: Graduate: 

Freshmen Masters 

Junior Ph.D. 

Senior Other: 

Mixed 

Other: 



3 How many times (e.g., classes) had you taught this particular group? 
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4. What was the topic of the class (general description)? 
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5. What were you trying to accomplish (i.e., teaching goal)? 



6. What did you want the students to learn (i.e., learning goal) 



7. What happened that made this incident memorable or critical? 



8. What did you do (i.e., your actions and strategies)? 



9. What were you thinking and feeling during this incident? 



10. What do you feel was most demanding about the incident? 



1 1 . What is it that you know that contributed to making this an exemplary incident? 



12. Did you learn anything from the experience? 
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